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Overview 

This paper discusses the process through which a powerful multidimensional measure 
of affect and cognition in relation to adult learning of computing skills was derived from its 
early theoretical stages to its validation using structural equation modeling. The discussion 
emphasises the importance of ensuring a strong substantive basis from which to develop 
reliable items for a measure, as well as the usefulness of gathering qualitative data both in the 
factor and item design stages to supplement the confirmatory factor analyses. 

Objectives 

The specific objectives of the research were: 

1. To design a reliable and valid instrument for measuring negative affect and cognitions in 
relation to learning computing skills whose multidimensional constructs were based on 
sound theoretical grounds and were informed by qualitative data. 

2. To demonstrate the invariance of the factor structure of the instrument across four different 
faculty groups. 

Theoretical Framework: Substantive and Methodological 

As Bandalos and Benson (1990 p. 51) point out, there has existed in the area of 
computer anxiety research for some time a degree of "inconsistency in the hypothesized 
dimensionality of the construct, computer anxiety. If specific dimensions are hypothesized to 
underlie computer anxiety, a clear explication of these dimensions may allow for a more 
nearly precise measurement of this construct". 

The aim of the present study, therefore, was to design a scale which did clearly 
explicate these dimensions and which would measure, with considerable validity and 
reliability, the multiple dimensions of computer anxiety in a training situation. This was 
achieved by incorporating into a model of computer anxiety for beginning adult learners a 
number of factors which had emerged reliably from previous anxiety research together with 
additional dimensions which were derived from theories of motivation and learning. 

Further insights about the nature and correlates of computer anxiety and negative 
cognitions were gleaned from three sources of qualitative data: the first, from interviews 
conducted with researchers who have designed and administered measures of anxiety; the 
second, from students who have undertaken introductory computer training courses and from 
their instructors; and the third, from those responsible for the design and administration of 
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Theoretical Framework: Substantive and Methodological 

As Bandalos and Benson (1990 p. 51) point out, there has existed in the area of 
computer anxiety research for some time a degree of "inconsistency in the hypothesized 
dimensionality of the construct, computer anxiety. If specific dimensions are hypothesized to 
underlie computer anxiety, a clear explication of these dimensions may allow for a more 
nearly precise measurement of this construct". 

The aim of the present study, therefore, was to design a scale which did clearly 
explicate these dimensions and which would measure, with considerable validity and 
reliability, the multiple dimensions of computer anxiety in a training situation. This was 
achieved by incorporating into a model of computer anxiety for beginning adult learners a 
number of factors which had emerged reliably from previous anxiety research together with 
additional dimensions which were derived from theories of motivation and learning. 

Further insights about the nature and correlates of computer anxiety and negative 
cognitions were gleaned from three sources of qualitative data: the first, from interviews 
conducted with researchers who have designed and administered measures of anxiety; the 
second, from students who have undertaken introductory computer training courses and from 
their instructors; and the third, from those responsible for the design and administration of 
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computer training courses in different university faculties. The qualitative approaches used in 
this study were based on the work of Bouma (1996), Cohen & Manion (1994), Miles and 
Huberman (1994), and Wiersma (1991). 

The dimensions included in the theoretical model were as follows: Competence with 
Computers; Handling Computer Equipment; Receiving Feedback on Computing Skills; and 
Learning about Basic Computer Functions; Positive Sense of Control (cognitions about being 
able to master computers); Fear (fearful cognitions about damaging the computer and public 
embarrassment in computing situations); Computing Self-Concept (perceptions of self-image 
in relation to computers); and, Worry, Happiness, Distractability, and Physiological 
Symptoms (of anxiety in computing situations). 

In the first instance, two factors defining Sense of Control were hypothesized: 
Positive Cognitions (or self-talk) about being able to master computers, and Negative 
Cognitions expressing fear about learning to use the computer. These were derived from the 
original Computer Thoughts Survey (CTS) of Rosen, Sears & Weil (1987a & b), and 
supported by the argument of Bagozzi (1994) who maintains that all the cognitive and 
affective elements of one's attitudes consist of aggregations of positive, negative, and positive 
and negative atomistic mental representations of knowledge (or knowledge units held in 
memory). In the second approach, a model positing the existence of one substantive factor 
plus a "negative item method effect" was tested (Marsh, 1994) on the premise that negatively 
worded items in a survey do not measure the opposite of the same construct as do positively 
worded items, and are, therefore, too "weak" to include in a total scale score. 

Contrary to the evidence in regard to positive and negative sense of control 
perceptions, the research literature on general self-concept strongly argues that, rather than 
representing a substantive negative factor (negative self-concept), negatively worded items in 
an instrument merely contain a measurement-method "artefact" (Benson & Hocevar, 1985; 
Marsh, 1986; 1994). Much of this literature, however, is derived from research with young 
children, while the present sample consisted of adults in a tertiary education setting and who 
were predicted, therefore, to be better able to cope with the cognitive demands of negatively 
worded items. The same approach to model testing was adopted in this scale as in the 
previous one. In other words, both a single factor and a two-factor model with regard to self- 
concept about one's proficiency with computers were tested. 

In the case of both the Sense of Control and Computing Self-Concept scales, items 
were written to assess different responses in relation to cognitions. These were expressed 
semantically in Sense of Control as positive and negative self-talk ("I can master the 
computer" or "Everyone else but me knows what they are doing"), and in the Computing Self- 
Concept scale as a positive or negative perceptions of self-image in relation to computing 
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situations ("I can help others solve computer problems" or " I am not the type to do well with 
computers"). 

In all, therefore, the theoretical sources for the constructs came from the research 
literature in three areas: 



1. Clinical anxiety, especially that relating to cognitive-behavioural training (Barlow, 1988; 
Beck & Emery, 1985; Michelson & Ascher, 1987); the impact of emotions on learning and 
achievement (Pekrun, 1992); and the effects of interfering self-doubts (Sarason, 1960, 
1972, 1984; Schwartzer, 1996). 

2. Educational psychology, notably self-efficacy theory (Bandura, 1977, 1986, 1988, 1993, 
1997) and self-worth theory of achievement motivation (Covington, 1984, 1992). 

3. Theoretical literature relating to computer anxiety (Rosen & Maguire, 1990; Rosen, Sears, 
' & Weil, 1993; Rosen, & Weil, 1995a &b). 

4. Pre-existing instruments of computer anxiety and attitudes (Loyd & Gressard, 1984; 
Maurer & Simonson, 1984; Heinssen, Glass & Knight, 1987; Rosen, Sears and Weil, 
1987a & b; Meier, 1988), as well as those measuring clinical anxiety (Sarason & Sarason, 
1987; Sarason, 1991; Spielberger, 1972a &b; 1975a & b; Spielberger, Pollans & Worden, 
1984). 



Method 

Participants 

The Computer Anxiety and Learning Measure (CALM) was validated using a sample 
(n = 794) of undergraduate students enrolled in four different faculties of a regional university 
in the Greater Western area of Sydney: Education, Health, Arts and Social Sciences, and 
Business and Technology. Approximately 70% of the sample came from an English speaking 
language background. The remaining 30% of the students came from a wide range of 
language backgrounds (42 languages), with the most common being Chinese (n=28), Greek 
(n=7), Italian (n=25), Spanish (n=20), Arabic (n=18), Vietnamese (n=16), Croation (n=12) 
and Lebanese (n=10). 170 males and 285 females had attended co-educational schools, while 
183 males and 98 females came from single-sex schools. A number of students moved 
between the two types of schools, and some respondents failed to indicate the type of school 
attended. Four hundred and forty-two respondents had attended government schools, and 268 
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had attended non-government schools. A small number moved between the two sectors of 
education. Four hundred and six respondents entered the university of the basis of their score 
on the Tertiary Entrance Rank (TER), the usual high school matriculation path, 255 entered on 
the basis of an alternative special entry test, while 102 entered on other criteria (such as 
overseas qualifications or mature age). There were missing data for approximately 31 
students. Participants came from families within the working and middle classes in terms of 
socioeconomic status. 

As can be seen from the above description, this sample was demographically diverse 
enough to be generalisable to other Western settings such as inner-city, industrial and non- 
affluent suburban areas. At the time that the CALM was administered, these students were 
about undertake a semester-long introductory computer training course in their respective 
faculties, required by the university for graduation. 

Measures 

Confirmatory factor analyses using LISREL 7 (Joreskog & Sorbom, 1989) were 
conducted on the a priori models of the CALM subscales. Preliminary exploratory factor 
analyses (SPPSX) had been conducted earlier to refine the original model. 

Subsequent to the tests of goodness-of-fit for the a priori models reported earlier, tests 
of factorial invariance were conducted to determine the stability and generalisability of the 
results across independent groups. 

As parallel data exist for each of the faculty groups in the sample, confirmatory factor 
analysis provides a particularly powerful test of the equivalence of solutions across these 
groups (for further discussion see Bollen, 1989; Marsh & Hocevar, 1985; Marsh, 1993, 1994). 
Using such analyses, it is possible to fit the data subject to the constraint that any one, any set, 
or all parameters are equal in the multiple groups. "The minimal condition for 'factorial 
invariance' is the equivalence of all factor loadings in the multiple groups (Marsh, 1994, p. 
11). Following Marsh's (1994) hierarchical ordering of invariance testing, three tests were 
conducted: 

• Model 1 : Total non-invariance with no between-groups invariance constraints. 

• Model 2 : Factor loadings invariant across groups. 

• Model 3 : Total invariance with all parameters (factor loadings, factor correlations, factor 

variances, and uniquenesses) constrained to be the same for all groups. 
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When the focus of the confirmatory factor analysis is to test invariance across multiple 
groups, it is advised that analyses be conducted using covariance matrices, not correlation 
matrices that are standardised in relation to responses by each group separately (for further 
discussion see Joreskog & Sorbom, 1989). In the present investigation, therefore, the 
invariance of factor solutions based on responses by students from three faculty groups were 
tested using covariance matrices. The three groups consisted of students from the faculties of 
Business and Technology, Arts and Social Sciences, and Education combined with a smaller 
sample of students in the Faculty of Health. 

The logic behind combining these samples was twofold: as the Faculty of Health had 
only 79 students compared with 165 in the Faculty of Education, and had a similar relative 
proportion of males to females as the Education group, (Education males = 17%; Health males 
= 20%), it was considered appropriate to combine the two so that factorial invariance tests 
could be conducted on sufficiently large samples to satisfy the requirements for optimal 
sample size needed for invariance testing using the LISREL program (that is, over 200 
students in each group). Secondly, both samples represented “helping” professions so that 
students enrolled in them were likely to be more similar than from other faculties. Because 
these three faculty group initial intakes are self-selected to be quite different, these tests of 
factorial invariance provide a good test of the generalisability of the factor solution of the 
Computer Anxiety and Learning Measure. As the group sizes were modest (Business and 
Technology = 282, Arts and Social Sciences = 268, and Education combined with Health = 
244) and were not optimal for using polychoric and/or asymptotic matrices (as used for testing 
the a priori models for the full group), it was necessary to run subsequent analyses using 
covariance matrices. The Tucker-Lewis Index (TLI) and Relative Noncentrality Index (RNI) 
were used to assess goodness-of-fit. 

In summary, therefore, analyses of the a priori models and the tests of invariance were 
conducted using different matrices. This was due both to differences in sample size between 
the full group (n = 794) and separate faculty groups (n = between 235 and 273), where 
asymptotic covariance matrices are appropriate for large samples but not for small), and to the 
fact that covariance matrices are required for invariance testing. 

Results 

An original 100 items were reduced to 65 items through a combination of exploratory 
and confirmatory factor analyses, as well as an evaluation of item face validity by three 
independent raters. Eleven first-order and two second-order factors were subsumed into four 
subscales: Gaining Initial Computing Skills, Sense of Control, Computing Self-Concept, and 
State Anxiety in Computing Situations. The overall goodness-of-fit indices (AGFI, RNI and 
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TLI) 1 for the models of each subscale ranged from .96 to .99. Root mean square residuals of 
below .05 (.047, .049, .034, and .048, respectively, for each of the four final subscales) 
provided additional indices of the goodness-of-fit of the models for each scale Furthermore, 
analyses of factorial invariance (Marsh, 1994) demonstrated that the model of computer 
anxiety and cognitions presented in the CALM can be applied with considerable confidence 
across different faculty groups on the basis of very good fit of the model (RNI and TLI 
ranging between .93 to .99). 

Separate faculty group analyses were conducted to test the model fit in each faculty. 
Irrespective of differences in academic focus between the faculty groups, it was shown that 
the goodness-of-fit for the models of anxiety in each scale were very comparable and 
relatively high, reaching the Tucker-Lewis index of acceptable fit (.90) for Faculties 1 and 3 
for the Sense of Control scale, and exceeding it considerably in most other cases. In sum, 
both the RNI and TLI indices were at least .9 for each of the three groups analysed separately. 
Table la presents the results for total group analyses for the a priori model based on 
asymptotic covariance matrices, and Table lb presents results of invariance tests of the 
models using covariance matrices (see Appendix 1) 



Gaining initial computing skills 

This scale was defined by four factors related to general anxiety about gaining initial 
computing skills: Learning about Basic Computer Functions; Competence with Computers; 
Handling Computer Equipment; and Receiving Feedback on Computing Skills. Four models 
were tested for the Gaining Initial Computing Skills scale: 

• Model 1 was the four factor model submitted to EFA using exactly the original 32 items. 

• Model 2, based on the results of Model 1, extracted those items which conformed to the 
criteria outlined above. This left 22 items. 

Using these 22 items, alpha coefficients for each factor were: 

Factor 1: .88 
Factor 2: .78 
Factor 3: .88 
Factor 4: .88 

As the results from Model 1 indicated a high correlation (mean r= > .6) between each of 
the four factors and as it was hypothesized that all items in the Gaining Initial Computing 



1 AGFI: Adjusted Goodness of Fit; RNI: Relative Non-Centrality Index; TLI: Tucker-Lewis Index. 
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Skills scale logically related to aspects of gaining initial computing skills in formal class 

settings, two additional models were tested. 

• Model 3, therefore, was a unidimensional one in which all items from the previous 
(second) model were included as one factor. 

• Model 4 predicted that there was one higher-order factor, general anxiety about Gaining 
Initial Computing Skills, in addition to four first-order factors (Learning about the Basic 
Functions of Computers; Competence with Computers; Handling Computer Equipment; 
and Receiving Feedback on Computing Competence). This model was believed to be the 
most appropriate fit as it was in keeping with the substantive grounds for the items in the 
Gaining Initial Computing Skills scale, and would also account for the correlation between 
factors, referred to earlier. 

Table 2 

Four Models of General Anxiety about Gaining Initial Computing Skills 



Goodness-of-Fit Indicators for Confirmatory Factor Analysis 



Model 


X 2 


df 


AGFI 


RMSR 


X 2 /df 

ratio 


RNI 


TLI 


Model 1 


1543.75 


438 


.935 


.057 


3.61 


.929 


.918 


Model 2 


569.44 


203 


.968 


.046 


2.81 


.976 


.968 


Model 3 


2294.85 


209 


.873 


.108 


10.99 


.841 


.824 


Model 4 


587.60 


205 


.967 


.047 


2.87 


.971 


.967 


Note 1. 
Model 1: 
Model 2: 


Four factors (32 items) 
Four factors (the “best” 


22 items) 











Model 3: One factor (22 items) 

Model 4: Higher-order factor plus four first-order factors 

Note 2. 

AGFI = adjusted goodness-of-fit index RMSR = root mean square residual 

TLI = Tucker-Lewis index RNI = relative noncentrality index 



If one were to use the chi-square/degrees of freedom ratio as the strongest indicator, 
the "best" fit to the data is Model 2 (ratio of 2.8) with the AGFI and TLI very strong at .968 
(see Table 2. Note : the large sample size inflates the chi-square along with the large number 
of parameters estimated). However, on substantive grounds, namely, the correlation between 
factors, Model 4 is preferred. The ehi-square/degrees of freedom ratio in this case is only 
marginally higher (ratio of 2.87) with the AGFI and TLI still very high at .967. Support for 



this model is also provided by the high correlation between the first-order factors (ranging 
from .65 to .94) as well as the target coefficient of .97 (the ratio of chi-square of the first-order 
factor model to the chi-square of the higher-order factor model), which indicates that the more 
restrictive higher-order model provides an excellent fit to the data. 

The correlations among the four a priori factors are positive and range between .63 and 
.84 (mean r = .75). While moderate, such correlations provide support for arguing the 
separation of the four factors in the Gaining Initial Computing Skills scale in that an average 
of 44% of the variance in the factors is unexplained. A higher-order model attempts to 
explain the correlations among the first-order factors. In this scale, the correlation between 
the higher-order factor and the first-order factors ranges between .78 and .94 (mean r = .87). 
Such a strong correlation supports the existence of an underlying factor which may justify the 
use of a total score in this scale. However, examination of the residuals indicates that from 
12% to 39% of the variance is unexplained by the higher-order model. It is clear that one 
cannot adequately summarise the four factors with a higher-order one. Thus, caution needs to 
be exercised when using a total score to recognise that, while providing a useful overview and 
demonstrating a logical substantive relationship between the first-order factors, such a score 
would be an oversimplification of an individual's anxiety in Gaining Initial Computing Skills. 
Far more information would be obtained by recognising the multidimensionality of this factor. 

While statistically significant, the substantive difference between the first-order and 
the higher-order model needs to be considered carefully (for detailed discussion of the 
evaluation of higher-order models see Marsh, 1987; Marsh & Hocevar, 1985). Clearly, there 
is virtually no difference in the goodness-of-fit indices (TLIs of .967 and .968). The residuals 
range between .06 and .22. 

As with the Gaining Initial Computing skills scale, LISREL 7 was used to perform 
confirmatory factor analyses on the data in the Sense of Control scale. Three models were fit 
to the data. 

• Model 1 used the factor structure derived from the initial substantive model from which 
items had been generated, and which had been supported by exploratory factor analyses, 
albeit somewhat "pruned". In this model, two substantive factors (positive and negative) 
were posited. 

• Model 2 examined the possibility that there was only one factor underlying both the 
negative and positive items, that is, it was unidimensional. 

• Model 3 predicted two factors, one "substantive" one on which all 12 items were allowed 
to load, and a "negative item" method factor for the 6 negative items only. Here the factor 
loadings for the two factors and the error/uniquenesses associated with each item were 
freely estimated. This allowed for all the negative items to load on one separate method 
factor. 

eric 1 1 



9 



Sense of control 

The two factors hypothesized to define this construct were Positive and Negative 
Cognitions (or self-talk) about being able to master computers. An alternative model was 
tested as well, that comprising one general factor plus a negative item artefactor. 

Table 3 

Three Models of Sense of Control 



Goodness-of-Fit Indicators for Confirmatory Factor Analysis 



Model 


X 1 


df 


AGFI 


RMSR 


X 2 /df 

ratio 


RNI 


TLI 


Model 1 


204.73 


53 


.976 


.490 


3.86 


.980 


.975 


Model 2 


1445.40 


54 


.833 


.144 


26.77 


.819 


.778 


Model 3 


161.86 


48 


.979 


.420 


3.37 


.985 


.980 



Note 1. 

Model 1: Two factors (12 items) 

Model 2: One factor only (12 items) 

Model 3: Two factors: one substantive factor and one method “artefactor” 

Note 2. 

AGFI = adjusted goodness-of-fit index RMSR = root mean square residual 

TLI = Tucker-Lewis index RNI = relative noncentrality index 



As seen in Table 3, Model 2 is clearly a very poor one with a chi-square/degrees of 
freedom ratio of 26.77, an AGFI of .833 RMSR of .144 and TLI of .778. Using the criteria of 
parsimony and the chi-square/degrees of freedom ratio as the strongest indicators, the "best" 
fit to the data is Model 3 (chi-sq/degrees of freedom ratio of 3.37) with the AGFI high at .979 
and TLI at .980. It is worth noting that Model 1, however, is only marginally weaker in a 
statistical sense, with an AGFI of .976 and a TLI of .975 (Note: the large sample size, along 
with the large number of parameters estimated, inflates the chi-square so that the AGFI and 
TLI are taken to be more appropriate indicators of fit). 

The relatively modest correlation (PHI = .52) between the latent factors, with only 
27% of their variation shared, however, also supports Model 1 substantively. The two-factor 
model of the Sense of Control scale (Model 1) was chosen on substantive grounds. 

Computing self-concept 

Exploratory factor analyses were performed on the 20 items in this scale (shown below 
in Table 4). 
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As with the previous scale, Sense of Control, it was originally hypothesized that there would 

be two factors in this scale; one which measured a Positive Computing Self-Concept ("I am 

good at computing"), and the other which measured a Negative Computing Self-Concept ("I 

am no good at computing"). 

LISREL 7 confirmatory factor analyses were performed on the Computing Self-Concept scale. 

Three models were fit to the data: 

• Model 1 predicted that there were two factors (positive and negative) as in the exploratory 
factor analyses in which the factors were allowed to be correlated. 

• Model 2 was a unidimensional factor estimation. 

• Model 3 estimated two factors: One a "real" or substantive factor; the other a "method" 
factor which accounts for the negatively worded items. Here factor loadings for each factor 
and the error/uniqueness associated with each item were freely estimated. This allowed for 
all the negative items to load on one separate method factor. 

Table 4 

Three Models of Computing Self-Concept 



Goodness-of-Fit Indicators for Confirmatory Factor Analysis 



Model 


x 2 


df 


AGFI 


RMSR 


X 2 /df 

ratio 


RNI 


TLI 


Model 1 


165.53 


43 


.978 


.041 


3.85 


.983 


.979 


Model 2 


387.71 


44 


.950 


.068 


8.80 


.953 


.941 


Model 3 


114.91 


39 


.983 


.034 


2.94 


.990 


.985 


Note 1 



Model 1: Two factors (1 1 items) 

Model 2: One factor only (1 1 items) 

Model 3: Two factors: one substantive factor and one method “artefactor” 

Note 2: 

AGFI = adjusted goodness-of-fit index RMSR = root mean square residual 

TLI = Tucker-Lewis index RNI = relative noncentrality index 



Table 4 shows that Model 3 provides an excellent fit to the data with an AGFI of .983 
and a TLI of .985. It is also the most parsimonious model. Furthermore, the chi-sq/degrees of 
freedom ratio is the lowest of the three models. 

Statistically, the evidence supports a model for the Computing Self-Concept scale of 
one self-concept factor and an additional "method effect" constituted by negatively worded 
items. This is supported by the relatively strong correlation between the two hypothesized 
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factors (PHI = .84), indicating that approximately 71% of the variation between the latent 
factors is shared. However, as with the Sense of Control scale, it is clear that there is little 
statistical difference between Model 3 and the two-factor (positive and negative), Model 1 , 
whose AGFI was .978 and whose TLI was .979. On substantive grounds again, the one-factor 
model is chosen in this case (see discussion earlier). 

State anxiety in computing situations 

The research literature in the area of anxiety of a clinical nature typically reports three 
component of anxiety: cognitive, emotional and somatic. These components, therefore, were 
hypothesised to comprise the state anxiety that individuals would experience in computing 
situations. Specifically, in the CALM instrument, the cognitive component was hypothesised 
to consist of two factors - Worry and Distractability; the emotional component was 
hypothesised to consist of one factor - Happiness; and the somatic component was 
hypothesised to consist of the Physiological Symptoms factor. 

Using LISREL 7, confirmatory factor analyses were conducted on the revised set of items 
from the exploratory factor analyses in the State Anxiety in Computing Situations scale. Two 
models were fit to the data: 

• Model 1 predicted that there were four first-order factors in the State Anxiety in 
Computing Situations scale, comparable to the exploratory factor analyses. 

• Model 2 predicted that, in addition to the four first-order factors in model one, there was 
one second- or higher-order factor. 

• 

Table 5 

Two Models of State Anxiety in Computing Situations 



Goodness-of-Fit Indicators for Confirmatory Factor Analysis 



Model 


X 2 


df 


AGFI 


RMSR 


X 2 /df 

ratio 


RNI 


TLI 


Model 1 


345.30 


164 


.986 


.045 


2.10 


.988 


.988 


Model 2 


372.55 


166 


.985 


.048 


2.24 


.987 


.987 



Note 1 

Model 1 : Four factors (20 items) 

Model 2: Higher-order factor plus four first-order factors 

Note 2, 

AGFI = adjusted goodness-of-fit index; RMSR = root mean square residual; 

TLI = Tucker-Lewis index; RNI = relative noncentrality index; 
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The four factor model clearly provides a very good fit to the data, as shown in Table 5. 
However, using the ratio of chi-square of the first-order factor model to the chi-square of the 
higher-order factor model as a criterion (Marsh and Hocevar, 1985), the value of the target 
coefficient (.93) indicates that the more restrictive factor model also provides a very good fit 
to the data. There was moderate to high correlation between the first-order factors (ranging 
from .43 to .8) which was considered sufficient to support the existence of a higher-order 
factor as well, although the Distractability factor had the lowest correlation with each of the 
other factors (ranging between .43 and .51). As with the Gaining Initial Computing Skills 
scale, there is virtually no statistical difference between the two models. The results clearly 
support the multidimensionality of this scale at the same time as providing evidence of the 
underlying substantive relationship among the four factors. The comparative weakness of the 
Distractability factor was recognised and efforts to remedy this were implemented, as 
mentioned above. 

Factor 1 (Worry) accounts for all of the variance in that scale (i.e., a higher-order 
factor loading of 1.00), while factor 3 (Distractability) accounts for only 30% of the variance 
(i.e., a higher-order factor loading of .53 . Factors 2 (Happiness) and 3 (Physiological 
symptoms) account for over half of the variance in those factors. Some of this inconsistency 
may be explained by the number of items in each factor, with factor 1 having 7 items, and 
factor 3 having only 2 items. 

It would appear that the Distractability construct only partly fits within a Worry state 
context, although the literature would support its very strong substantive connection. As 
Wine (1971) and Heckhausen (1991) have demonstrated, the negative effects of worry on 
performance are seen in terms of task irrelevant responses such as interfering self-related 
thoughts. From a substantive point of view, therefore, it would be valuable to be able to 
determine an individual's overall state of anxiety in a computing context, and examine scores 
on each of the cognitive, emotional and somatic components of state anxiety. 

The final Computer Anxiety and Learning Measure (CALM) that was derived from 
both EFAs and CFAs in the present study comprised 65 items. Eleven first-order factors 
provided a good fit to the data. Two second-order factors explained the correlations between 
the four oblique factors in the Gaining Initial Computing Skills and State Anxiety in 
Computing Situations scales, respectively. In summary, the latent constructs in the model of 
computer anxiety that have evolved from the present research can be represented in the 
following way, as shown in Table 6. 

Clearly, with Tucker-Lewis indices for each scale of the Computer Anxiety and 
Learning Measure ranging between .968 and .988 (see Appendix 1, Table la), the support for 
the a priori models is very strong indeed. As can be seen in Table lb (see Appendix lb), 
conclusions based on the Tucker-Lewis index of goodness-of-fit indicate that the best fit for 
each scale is for models imposing complete invariance - the most parsimonious and, therefore, 
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those favoured by the Tucker-Lewis index (which penalises model complexity). There was 
considerable consistency between results for the three invariance models. Such strong support 
for invariance provides excellent support for all of the a priori models. 

In conclusion, therefore, it appears that the model of computer anxiety presented in the 
Computer Anxiety and Learning Measure can be applied with considerable confidence across 
different faculty groups on the basis of the very good fit of the invariance model. 



Table 6 



Factors Defining Computer Anxiety for Adult Learners Using the Computer Anxiety 
and Learning Measure: A Model of CALM 



Gaining Initial Computing Skills: 

Four first-order factors - 

* Learning about Basic Computer Functions 

* Competence with Computers; 

* Handling computer Equipment; 

* Receiving Feedback on Computing Skills 

One second-order factor - 

* Gaining Initial Computing Skills 



Sense of Control: 

One factor - 

* Sense of Control plus negative-item artefactor, or 
Two factors - 

* Positive Sense of Control and Fear (alternative model) 



Computing Self-Concept: 

One factor - 

* Computing Self-Concept plus negative-item artefactor, or 
Two factors - 

* Positive and Negative Computing Self-Concept (alternative model) 



State Anxiety in Computing Situations: 

Four first-order factors - 

* Worry 

* Happiness 

* Distractability 

* Physiological Symptoms 

One Second-order factor - 

* State Anxiety in Computing Situations 



The final version of CALM that has emerged from the exploratory and confirmatory 
factor analyses described in this paper is shown in Figure 1. The factor loadings and 
reliabilities derived from the LISREL 7 analyses for the sixty-five items retained in the final 
instrument are shown in Table 7 (see Appendix 2). 

In conclusion, although the Computer Anxiety and Learning Measure appears 
statistically and substantively “robust”, the generalisability of the model of computer anxiety 
presented in this study has yet to be demonstrated. While the original sample was 
representative of the composition of the population of interest, namely, adults undertaking 
initial computer literacy skills training courses, the model needs to be replicated with a more 
diverse sample and with a non-student population. It is important to note, however, that the 
tests of factorial invariance across different faculties reported earlier give considerable support 
for the stability and generalisabilty of the findings. Future research would need to examine 
whether the CALM model is invariant across different groups in similar learning /training 
environments (cf. Marcoulides, Mayes & Wiseman, 1995). 

Discussion 

Results from the multimethod research techniques used in the present study provide 
strong support for the validity and reliability of the Computer Anxiety and Learning Measure 
(CALM) of the multifaceted nature of computer anxiety, and for the invariance of its 
multidimensional structure across different groups. This measure, therefore, appears to have 
wide applicability for the assessment of specific facets of anxiety and negative cognitions of 
adult learners undertaking computer training courses. As such, the CALM has the potential to 
inform strategies for instruction and remediation on the basis of diagnosis of individual 
differences in these constructs. 

The strongest goodness-of-fit indices from the confirmatory factor analyses reported in 
the present study support a multifaceted model of computer anxiety for beginning adult users 
which consists of ten factors plus one negative item factor. Such a model provides excellent 
explanatory power in that the use of individual facets of a construct can sometimes predict 
dependent variables more accurately than the more general construct (Carver, 1989). A more 
parsimonious model still, consisting of only five factors plus a measurement-method effect, 
was identified using CFA analyses of the data. In particular, the results supported the 
existence of one second (higher) order factor for each of the Gaining Initial Computing Skills 
and State Anxiety in Computing Situations scales. This was not surprising, as all items 
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defining the first-order factors in these scales related logically to the general construct for each 
section, as discussed previously. 

Although the results for the Gaining Initial Computing Skills and the State Anxiety in 
Computing Situations scales, based on their respective four factor models, support the 
multidimensional nature of these computer anxiety constructs, it is implied by the more 
parsimonious higher-order models in each case that it may be valid to subsume the different 
dimensions under a broader construct of General Anxiety in Gaining Initial Computing Skills. 
Thus, for administration of the CALM instrument in settings where the greater explanatory 
power of the four factor models of these scales is not deemed significant, the use of a single 
score for these single higher-order constructs may be justified (cf. Shek, 1993), while 
acknowledging the considerable oversimplification that this represents, as discussed earlier 
(because it is comparable to adopting a one factor model whose fit was considerably poorer). 
On the other hand, given that the overall fits for both first- and second-order models in these 
scales are almost identical (as shown in Table 2), the decision to adopt one or the other is 
dependent on one's research purpose. 

Conclusion 

While the Computer Anxiety and Learning Measure appears statistically and 
substantively “robust”, the generalisability of the model of computer anxiety presented in this 
study has yet to be demonstrated. While the original sample was representative of the 
composition of the population of interest, namely, adults undertaking initial computer literacy 
skills training courses, the model needs to be replicated with a more diverse sample and with a 
non-student population. It is important to note, however, that the tests of factorial invariance 
across different faculties reported earlier give considerable support for the stability and 
generalisabilty of the findings. Future research would need to examine whether the CALM 
model is invariant across different groups in similar learning /training environments (cf. 
Marcoulides, Mayes & Wiseman, 1995). 
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Final Model of the Computer Anxiety and Learning Measure (CALM) 
Derived from Exploratory and Confirmatory Factor Analyses 



Appendix 1 
Table la 



Summary of Goodness-of-fit Indices for "Best" Models of Scales in the 
Computer Anxiety And Learning Measure; Total Group Analyses 



Model 


Z 2 


df 


X 2 /df 


RNI 


TLI 


Gaining Initial Computing Skills 










null 


13332.17 


231 


57.715 


.000 


.000 


1 storder 


569.44 


203 


2.805 


.976 


.968 


hiorder 


587.60 


205 


2.866 


.971 


.967 


Sense of Control 












null 


7743.47 


66 


117.325 


.000 


.000 


methodeff 


459.16 


48 


9.566 


.985 


.980 


2 factor 


204.73 


53 


3.862 


.980 


.975 


Computing Self-Concept 










null 


7346.55 


55 


133.574 


.000 


.000 


methodeff 


114.91 


39 


2.946 


.990 


.985 


2 factor 


165.53 


43 


3.849 


.983 


.979 


State Anxiety in 


Computing Situations 








null 


16374.98 


190 


86.184 


.000 


.000 


1 storder 


345.30 


164 


2.10 


.988 


.988 


hiorder 


372.55 


166 


2.244 


.987 


.987 



Note. 

null = null models for the total group; lstorder = first-order models for the total group; 
hiorder = higher-order models for the total group; methodeff = method effect models for the 
total group; 2factor = two-factor models for the total group; TLI = Tucker-Lewis Index; 
RNI = relative noncentrality index. 
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Table lb 

Three-Group Analyses of Invariance for Faculty of Education/ Health: 
Faculty of Arts and Social Sciences; and Faculty of Business and Technology 



Model 


X 2 


df 


X 2 /df 


RNI 


TLI 


Gaining Initial Computing Skills (Higher order model) 






3gp (no inv) 


1570.34 


615 


2.553 


.934 


.975 


3gp(flinv) 


1593.93 


651 


2.448 


.935 


.977 


3gp (tot inv) 


1681.29 


711 


2.365 


.933 


.978 


Sense of Control (Two factor model) 








3gp (no inv) 


372.01 


144 


2.583 


.971 


.987 


3gp (fl inv) 


402.05 


180 


2.234 


.971 


.990 


3gp (tot inv) 


423.40 


204 


2.075 


.972 


.991 


Computing Self-Concept (Method effect model) 






3gp (no inv) 


283.62 


117 


2.424 


.979 


.990 


3gp (fl inv) 


324.02 


149 


2.175 


.977 


.992 


3gp (tot inv) 


353.18 


171 


2.065 


.977 


.992 


State Anxiety in Computing Situations (Higher order model) 




3gp (no inv) 


956.09 


498 


1.920 


.973 


.990 


3gp (fl inv) 


993.03 


530 


1.874 


.974 


.990 


3gp (tot inv) 


1091.44 


586 


1.863 


.971 


.990 



Note. 

3gp (no inv) = no invariance constraints across the three groups; 3gp (fl inv) = refers to factor 
loadings invariant across the three groups; 3gp (tot inv) = refers to all parameters constrained to 
be invariant across the three groups; TLI = Tucker-Lewis Index; RNI = relative noncentrality 
index. 
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Appendix 2 
Table 7 



Standardised Estimates for the 65 item Computer Learning and Anxiety Measure 





Item 

loading 


factor 


SMC 


GAINING INITIAL COMPUTING SKILLS 






SCALE 1: COMPETENCE ANXIETY 






3. 


Taking a test on my computer competence 


.79 


.62 


4. 


Working in a job that requires some computer experience 


.73 


.53 


6. 


Getting "error" messages from the computer 


.66 


.40 


21. 


Teaching someone else about computers 


.74 


.55 


26. 


Dealing with computer malfunctions 


.81 


.66 


29. 


Being evaluated on my computer competence 


.84 


.71 


33. 


Learning about computers without structured guidance 


.72 


.52 


SCALE 2: EQUIPMENT ANXIETY 






14. 


Using computerised equipment 


.80 


.65 


28. 


Printing off documents 


.79 


.63 


30. 


Using a "mouse" 


.62 


.40 


35. 


Presenting work completed on a computer 


.75 


.56 


SCALE 3: FEEDBACK ON COMPUTING SKILLS 






37. 


Being taught how to use a computer by a peer 


.75 


.56 


38. 


Getting feedback from my teacher on my computer skills 


.93 


.89 


39. 


Getting feedback from my peers on my computer skills 


.88 


.77 


40. 


Collaborating with a friend while learning to use a computer 


.79 


.63 


41. 


Getting feedback on my computer skills 


.93 


.87 


SCALE 4: LEARNING ABOUT BASIC COMPUTER FUNCTIONS 






1. 


Taking a course in a computer language 


.72 


.52 


11. 


Learning computer terminology 


.81 


.66 


12. 


Reading a computer manual. 


.68 


.46 


15. 


Learning how a computer works 


.83 


.69 


24. 


Learning the operating system of a computer 


.89 


.79 


25. 


Learning a new computer application 


.90 


.82 



STATE ANXIETY IN COMPUTING SITUATIONS 



SCALE 1: WORRY 



4. 


Feelings of unease 


.86 


.75 


10. 


Worried about possible problems 


.70 


.49 


18. 


Threatened 


.85 


.72 


20. 


Insecure 


.89 


.78 


21. 


Helpless 


.81 


.66 


27. 


Worried 


.85 


.73 


28. 


Rattled 


.75 


.57 
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Item 


factor 

loading 


SMC 


SCALE 2: HAPPINESS 


14. Happy 


.81 


.65 


15. Comfortable 


.90 


.81 


19. Secure 


.89 


.80 


26. Relaxed 


.92 


.84 


29. At ease 


.91 


.83 


30. Content 


.87 


.76 


SCALE 3: PHYSIOLOGICAL SYMPTOMS 


1 . Nervous stomach," butterflies" 


.91 


.83 


2. Hot and sweaty 


.89 


.80 


3. Heart palpitations 


.91 


.83 


12. Dry mouth 


.80 


.63 


13. Sweaty palms 


.80 


.64 


SCALE 4: DISTRACT ABILITY 


5 . Lack of concentration 


.99 


.97 


6. Distractable 


.80 


.64 



SENSE OF CONTROL (Two-factor model) 






SCALE 1: POSITIVE SENSE OF CONTROL 


1 . I can master the computer 


.66 


.43 


7. I know I can do it 


.73 


.53 


16. I will be able to get the computer to do what I want 


.75 


.56 


18. I will understand what to do 


.81 


.66 


19. I feel in control of what I do 


.93 


.86 


20. I feel confident about my ability with computers 


.88 


.76 


3. Everyone else but me knows what they are doing 


.77 


.60 


SCALE 2: FEAR 


5. People will notice if I make a mistake 


.80 


.63 


9. I'm afraid I'll wreck the program 


.72 


.53 


1 2 . What if I hit the wrong key? 


.74 


.55 


13. I'm too embarrassed to ask for help 


.68 


.46 


17. I might break the machine 


.75 


.56 



SENSE OF CONTROL (One factor plus negative item method effect) 



1 . 


I can master the computer 


.66 


.43 


7. 


I know I can do it 


.73 


.53 


16. 


I will be able to get the computer to do what I want 


.75 


.56 


18. 


I will understand what to do 


.81 


.66 


19. 


I feel in control of what I do 


.93 


.86 


20. 


I feel confident about my ability with computers 


.88 


.76 




25 



Item 


factor 

loading 


** 


SMC 


3. Everyone else but me knows what they are doing 


.48 


.56 


.55 


5. People will notice if I make a mistake 


.41 


.69 


.64 


9. I’m afraid I’ll wreck the program 


.34 


.67 


.56 


12. What if I hit the wrong key? 


.37 


.65 


.56 


13. I’m too embarrassed to ask for help 


.31 


.63 


.49 


17. I might break the machine 


.37 


.66 


.57 



COMPUTING SELF-CONCEPT (Two-factor modeH 



SCALE 1: POSITIVE COMPUTING SELF-CONCEPT 








6. I am very confident when it comes to working with computers 


.89 




.79 


7. I can get good grades in computer courses 


.78 




.62 


15. Iam confident storing important information on a computer 


.78 




.60 


16. Iam sure I could solve any problems I had while using a computer 


.79 




.63 


17. I can help others solve computer problems 


.86 




.74 


19. I am sure that I can help others learn to use the computer 


.87 




.75 


SCALE 2: NEGATIVE COMPUTING SELF-CONCEPT 
1 . I am no good with computers 


.87 




.76 


4. I am not the type to do well with computers 


.85 




.72 


5 . I think using a computer would be very hard for me 


.81 




.66 


9. I don’t think I could handle a computer course 


.69 




.48 


11. I avoid using computers as much as possible 


.79 




.62 


COMPUTING SELF-CONCEPT (One factor olus negative item method effect") 


** 




6. Iam very confident when it comes to working with computers 


.89 




.79 


7. I can get good grades in computer courses 


.78 




.61 


15. Iam confident storing important information on a computer 


.78 




.60 


16. Iam sure I could solve any problems I had while using a computer 


.79 




.63 


17. I can help others solve computer problems 


.86 




•74 


19. I am sure that I can help others learn to use the computer 


.87 




.75 


1 . I am no good with computers 


.78 


.29 


.69 


4. Iam not the type to do well with computers 


.70 


.51 


.76 


5. I think using a computer would be very hard for me 


.66 


.55 


.73 


9. I don’t think I could handle a computer course 


.57 


.44 


.51 


11. I avoid using computers as much as possible 


.69 


.35 


.60 



Note 1 , factor loadings = factor loadings derived from confirmatory factor analysis (LISREL 7) 

SMC = squared multiple correlations (a measure of item reliability). 

Note 2 . The items in the Sense of Control and Computing Self-Concept scales were presented originally in 
random order. They are separated here into positively and negatively worded items. 

Note 3 . ** = For the negatively worded items in the Sense of Control and Computing Self-Concept scales the 

factor loadings on a separate negative-item method factor are shown in bold next to the item loadings 
on the factor as a whole. 
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